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Abstract 

We  propose  two  related  equilibrium  refinements  for  voting  and  agenda-setting  games,  Sequentially 
Weakly  Undominated  Equilibrium  (SWUE)  and  Markov  Trembling  Hand  Perfect  Equilibrium  (MTHPE), 
and  show  how  these  equilibrium  concepts  eliminate  non-intuitive  equilibria  that  arise  naturally  in  dynamic 
voting  games  and  games  in  which  random  or  deterministic  sequences  of  agenda-setters  make  offers  to 
several  players.  We  establish  existence  of  these  equilibria  in  finite  and  infinite  (for  MTHPE)  games, 
provide  a  characterization  of  the  structure  of  equilibria,  and  clarify  the  relationship  between  the  two 
concepts.  Finally,  we  show  how  these  concepts  can  be  applied  in  a  dynamic  model  of  endogenous  club 
formation. 
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1      Introduction 

In  many  political  economy  problems,  including  models  of  legislative  bargaining  (e.g.,  Baron, 
Diermeier,  and  Fong,  2008,  Battaglini  and  Coate,  2007,  2008,  Diermeier  and  Fong,  2009,  Dug- 
gan  and  Kalandrakis,  2007,  Romer  and  Rosenthal,  1978)  and  models  of  political  coalition  for- 
mation (e.g.,  Acemoglu,  Egorov,  and  Sonin,  2008),  agents  participate  in  multiple  rounds  of 
proposal  making  and  strategic  voting.  Noncooperative  game-theoretic  formulations  of  these 
models  typically  specify  the  extensive  form  of  the  game,  in  particular,  the  procedures  through 
which  proposals  are  made  and  the  voting  protocols  (e.g.,  whether  voting  is  sequential  or  simula- 
taneous) .  These  "details"  as  well  as  the  exact  notion  of  equilibrium  often  have  a  major  impact 
on  the  set  of  equilibria  and  may  introduce  "non-intuitive"  equilibria  in  many  of  these  games. 
This  is  a  significant  challenge  for  applied  analysis  in  this  area.  It  would  be  desirable  for  features 
of  the  game  that  appear  unimportant  and  difficult  to  map  to  reality  (such  as  the  exact  order  of 
proposals  and  the  details  of  the  voting  protocol)  not  to  have  a  major  impact  on  predictions,  and 
for  non-intuitive  equilibria  not  to  emerge  as  predictions.  Therefore,  it  appears  important  for 
political  economy  theory  to  have  equilibrium  concepts  that  (i)  can  be  applied  to  a  wide  variety 
of  political  situations;  (ii)  rule  out  non-intuitive  equilibria;  and  (iii)  make  broad  predictions 
that  are  independent  of  the  procedural  details  specified  in  the  extensive  form  of  the  political 
game.  The  next  example  illustrates  how  non-intuitive  equilibria  can  emerge  in  even  the  simplest 
political  games. 

Example  1  Consider  three  individuals,  each  strictly  preferring  option  a  to  b.  One  of  these  two 
options  will  be  implemented  by  voting.  Voting  is  simultaneous;  the  option  that  receives  the 
majority  will  be  implemented.  It  is  straightforward  to  verify  that  the  natural  equilibrium,  in 
which  all  three  individuals  vote  for  option  a,  is  a  Nash  equilibrium.  However,  the  non-intuitive 
possibility  that  all  three  individuals  vote  for  option  b  is  also  a  Nash  equilibrium.  When  any  two 
players  vote  for  b,  it  is  a  weak  best  response  for  the  third  one  to  do  so  as  well.  Naturally,  we  do 
not  think  that  all  three  individuals  supporting  option  6  is  a  reasonable  prediction. 

The  non-intuitive  prediction  implied  by  the  notion  of  Nash  equilibrium  in  the  previous  ex- 
ample is  generally  ignored,  either  by  using  one  of  two  refinements,  Nash  equilibria  in  (weakly) 
undominated  strategies  or  trembling-hand  perfect  equilibria  (THPE),  or  by  changing  the  game 
so  that  instead  of  simultaneous  voting  there  is  sequential  voting.  It  is  evident  why  the  first 
approach  would  work  as  voting  for  6  is  a  weakly  dominated  strategy  for  each  player.  The  second 
approach  also  works  for  relatively  intuitive  reasons:  with  sequential  voting,  the  game  becomes 
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dynamic,  and  it  is  easy  to  show  that  in  any  subgame-perfect  equilibrium  (SPE)  at  least  two  play- 
ers vote  for  a,  so  option  a  is  chosen.  Moreover,  this  result  applies  regardless  of  the  order  in  which 
the  three  individuals  vote.  Nevertheless,  sequential  voting  has  the  potential  disadvantage  that 
in  practice  most  votes  are  not  cast  sequentially,  but  simultaneously  (and  with  secret  ballots). 
Therefore,  this  approach  "works,"  though  it  does  so  by  changing  the  environment  to  one  that 
is  a  worse  approximation  to  the  situation  being  modeled.  Moreover,  this  discussion  highlights 
that  the  set  of  predictions  might  crucially  depend  on  seemingly  unimportant  procedural  details 
such  as  whether  voting  is  sequential  or  simultaneous. 

In  addition,  while  these  two  refinements  eliminate  non-intuitive  equilibria  in  static  voting 
games,  many  interesting  political  economy  problems,  such  as  those  mentioned  at  the  beginning, 
involve  dynamic  proposal  making  and  voting.  In  this  case,  these  refinements  do  not  necessarily 
rule  out  non-intuitive  equilibria.  This  is  illustrated  in  the  next  example. 

Example  2  Consider  the  same  environment  as  in  Example  1,  except  that  voting  takes  place 
at  two  dates,  1  and  2.  If  a  unanimity  of  voters  prefers  one  of  the  two  options  in  period  1,  then 
this  option  is  implemented.  Otherwise,  there  is  a  majority  vote  at  date  2,  identical  to  that  in 
Example  1.  In  addition,  we  assume  that  delay  is  costly.  Now  because  the  game  is  multi-stage, 
the  relevant  equilibrium  concept  is  SPE.  It  can  be  verified  that  all  three  individuals  voting  for 
option  b  in  both  dates  is  an  SPE.  This  is  because  all  three  players  voting  for  b  is  an  equilibrium 
in  the  subgame  starting  from  date  2.  Then,  expecting  option  b  being  implemented  at  the  date 
2,  all  players  would  be  best  responding  by  voting  for  b  at  date  1.  It  can  also  be  verified  that 
eliminating  weakly  dominated  strategies  does  not  change  this  outcome,  since  voting  for  option 
b  at  date  1  is  not  weakly  dominated  because  of  costly  delay. 

Example  2  shows  that  simply  eliminating  weakly  dominated  strategies  is  not  sufficient  to 
resolve  the  problems  that  arise  in  dynamic  voting  games.  Nevertheless,  it  can  also  be  used 
to  illustrate  that  the  sequential  elimination  of  weakly  dominated  strategies  is  typically  more 
powerful.  For  example,  the  strategy  "vote  for  b  in  both  periods"  is  weakly  dominated  by  the 
strategy  "vote  for  b  in  period  1 ,  vote  for  a  in  period  2" ,  and  the  strategy  "vote  for  a  in  period 
1,  for  6  in  period  2"  is  weakly  dominated  by  "vote  for  a  in  both  periods".  After  these  weakly 
dominated  strategies  are  eliminated,  the  strategy  "vote  for  b  in  period  1  and  a  in  period  2" 
becomes  weakly  dominated  by  "vote  for  a  in  both  periods"  in  the  reduced  game.  Note  that  this 
process  of  sequential  elimination  requires  that  all  weakly  dominated  strategies  are  eliminated 
simultaneously.^    We  can  strengthen  the  point  even  further  and  show  that  even  allowing  for 


^See  Duggan  (2003)  for  a  discussion  of  problems  with  this  equilibrium  refinement. 


sequential  elimination  of  weakly  dominated  strategies  may  not  always  rule  out  non-intuitive 
equilibria  (see  Example  5  in  the  Appendix). 

As  already  mentioned  above,  a  powerful  equilibrium  concept  that  often  eliminates  non- 
intuitive  predictions  is  trembling-hand  perfect  equilibria  (THPE).  However,  the  use  of  trembling- 
hand  perfection  concept  in  dynamic  games  requires  some  caution  (e.g.,  Osborne  and  Rubinstein, 
1994,  pp.  246-252).  One  way  to  define  the  THPE  in  dynamic  games  is  to  aUow  for  correlated 
trembles  if  a  player  has  a  move  at  different  stages  of  the  game;  a  more  restrictive  generalization 
is  to  focus  on  uncorrelated  trembles.  Example  6  in  the  Appendix  shows  that  neither  of  these 
generalizations  is  sufficient  to  eliminate  non-intuitive  equilibria  in  certain  dynamic  voting  games. 

Motivated  by  these  problems,  this  paper  introduces  two  simple,  easy-to-use  and  intuitive 
equilibrium  concepts  for  agenda-setting  games:  the  Sequentially  Weakly  Undominated  Equi- 
librium (SWUE)  and  Markov  Trembling  Hand  Perfect  Equilibrium.  (MTHPE).  We  refer  to  a 
perfect-information  game  as  an  agenda- setting  game  if  at  each  stage  of  this  game  either  one 
of  the  players  (or,  possibly.  Nature)  moves,  or  there  is  a  voting  among  a  subset  of  players  (a 
formal  definition  is  provided  below).  Dynamic  voting  games,  in  which  there  are  several  stages 
of  voting  over  proposals,  are  a  special  case  of  agenda-setting  games.  In  addition,  most  dynamic 
multilateral  bargaining  games  can  be  cast  as  agenda-setting  games.  In  this  context,  SWUE  is 
an  intuitive  generalization  of  the  idea  of  eliminating  weakly  dominated  strategies,  and  uses  this 
procedure  iteratively  starting  from  the  terminal  subgames.  We  prove  existence  of  SWUE  in 
finite  games.  One  disadvantage  of  SWUE  is  that  it  is  difficult  to  generalize  or  apply  to  infinite 
games.  Another,  and  a  more  serious  one,  is  that  it  does  not  necessarily  rule  out  all  non-intuitive 
equilibria,  as  Example  5  in  the  Appendix  shows.  Our  alternative  notion,  MTHPE,  which  fo- 
cuses on  Markovian  trembles,  has  three  desirable  features.  First,  it  eliminates  all  non-intuitive 
equilibria  in  agenda-setting  games.  Second,  it  can  be  applied  easily  and  straightforwardly  for  a 
larger  class  of  games,  including  infinite  games  with  finite  number  of  non-equivalent  subgames. 
Third,  we  will  provide  a  tight  characterization  of  the  structure  of  MTHPE  in  agenda-setting 
games  and  show  that  several  "details,"  such  as  whether  voting  is  simultaneous  or  sequential,  do 
not  affect  the  predictions  in  these  games.  In  particular,  we  will  show  that  the  set  of  MTHPE  of 
agenda-setting  games  with  simultaneous  voting  is  identical  to  the  set  of  MTHPE  (and  of  SPE) 
of  the  equivalent  sequential  voting  games.  This  result  is  also  useful,  since  games  with  sequential 
voting,  though  admittedly  less  realistic,  allow  a  straightforward  characterization  of  SPEs  (and 
we  will  show  of  MTHPEs)  by  backward  induction." 


^We  are  not  aware  of  any  other  equilibrium  concept  or  refinement  that  has  these  three  desirable  features. 


We  will  first  establish  the  existence  of  MTHPE  for  a  broad  class  of  games,  including  all 
agenda-setting  games,  and  then  provide  a  characterization  of  the  set  of  MTHPE  for  agenda- 
setting  games.  While  MTHPE  and  SWUE  are  not  subsets  of  one  another  in  general,  we  show 
that  for  agenda-setting  games  (and,  more  broadly,  for  finite  games)  an  MTHPE  is  always  a 
SWUE  (so  within  this  class  of  games  MTHPE  is  a  stronger  concept). 

Our  main  characterization  result  as  well  as  our  result  on  the  equivalence  of  the  set  of  equi- 
libria in  simultaneous  and  sequential  voting  games  rely  on  the  notion  of  regularization  which  we 
introduce.  A  regularization  replaces  each  stage  of  a  dynamic  game  where  multiple  players  vote  si- 
multaneously by  a  sequence  of  actions  (stages)  in  which  one  player  moves  at  a  time.  This  implies 
that  whenever  several  players  move  simultaneously,  there  are  many  regularizations.  However,  all 
the  resulting  games  have  the  same  set  of  equilibrium  payoffs.  A  key  part  of  our  characterization 
result  is  that  the  set  of  MTHPE  of  the  initial  (simultaneous  voting)  game  is  equivalent  to  the 
set  of  SPE  of  any  regularization.  This  result  both  implies  that  which  regularization  is  chosen 
is  immaterial  and  shows  that  computing  the  set  of  MTHPE  is  generally  quite  straightforward 
and  can  be  done  by  choosing  an  arbitrary  regularization  (an  arbitrary  sequence  of  voting  stages 
corresponding  to  the  original  game).  Note,  however,  that  the  Markovian  trembles  restriction 
in  MTHPE  is  critical  for  these  results:  Example  6  in  the  Appendix  shows  that  the  above  cor- 
respondence between  the  equilibria  of  simulataneous  voting  game  and  its  regularization  breaks 
down  when  we  use  the  standard  THPE  concept  instead  of  MTHPE. 

Finally,  we  show  how  either  or  both  concepts  can  be  applied  to  a  dynamic  voting  model 
proposed  and  analyzed  in  Acemoglu,  Egorov  and  Sonin  (2008)  under  sequential  voting.  This 
game  can  be  interpreted  as  one  of  elimination  (as  in  Soviet  Politburo)  or  as  a  game  of  dynamic 
coalition  formation,  similar  to  previous  pioneering  work  in  this  area,  Roberts  (1999)  and  Bar- 
bera,  Maschler,  and  Shalev  (2001).  In  Roberts  (1999),  the  club  membership  changes  subject  to 
approval  by  a  majority  of  the  current  members;  in  Barbera,  Maschler,  and  Shalev  (2001),  any 
member  of  the  club  might  unilaterally  admit  a  new  member  to  the  club.^  Although  Section  4 
applies  our  main  results  to  a  game  of  dynamic  coalition  formation,  our  equilibrium  concept  can 
also  be  straightforwardly  applied  to  dynamic  legislative  bargaining  games.  For  example,  recent 
interesting  work  by  Diermeier  and  Fong  (2009)  studies  legislative  bargaining  with  persistent 
agenda-setting  power  and  uses  a  solution  concept  that  is  a  special  case  of  MTHPE.  Applying 
MTHPE  in  their  model  would  give  identical  results. 


^See  also  Jack  and  Lagunoff  (2006)  and  Granot,  Maschler,  and  Shalev  (2002),  where  unanimity  is  required  for 
admission  to  a  club. 


Although  the  hterature  on  voting  games  is  vast  (see  Austen-Smith  and  Banks,  1999,  for  a 
modern  treatment),  the  notions  of  MTHPE  and  SWUE  are  new.  Two  previous  contributions 
are  particularly  noteworthy.  Moulin  (1979)  pioneered  application  of  dominance  solvability  to 
political  games.  Duggan  (2003)  discusses  the  possibility  of  non-intuitive  equilibria  in  political 
games,  suggests  several  solutions  and  also  contains  a  list  of  related  open  problems. 

The  rest  of  the  paper  is  organized  as  follows.  Section  2  introduces  the  concepts  of  Markov 
Trembling-Hand  Perfect  Equilibrium  and  Sequentially  Weakly  Undominated  Equilibrium.  Sec- 
tion 3  contains  the  main  results,  while  Section  4  provides  an  application  of  MTHPE.  Section  5 
concludes  and  the  Appendix  contains  two  additional  examples. 

2     Setup 

2.1      Setup  and  Notation 

Consider  a  general  n-person  T-stage  game  (where  T  is  a  natural  number  or  oo),  where  each 
individual  can  take  an  action  at  every  stage.  Let  the  action  profile  of  each  individual  i  be 

a*  =  (a*! , . . . ,  a^)   for  i  =  1 , . . . ,  n, 

with  al  S  Al  and  a*  G  A'  =  nf=i  ^t-  Let  ht  =  (ai, . . . ,  at)  be  the  history  of  play  up  to  stage 
t  (not  including  stage  t),  where  a^  =  (a^, . . . ,  a"),  so  ^o  is  the  history  at  the  beginning  of  the 
game,  and  let  Ht  be  the  set  of  histories  ht  for  t  :  0  <  t  <T~1.  We  denote  the  set  of  all  potential 
histories  up  to  date  t  by  if  *  =  IJs=o  ^«  •  ^^^  i-continuation  action  profiles  be 


a'-*  =  (aj,  aj+i, . . . ,  a^)   for  i  =  1, . . . , 


n. 


with  the  set  of  continuation  action  profiles  for  player  i  denoted  by  A*'*.   Symmetrically,  define 
t-truncated  action  profiles  as 


a 


L-t 


[a\,a\,...,a\_i)   for  i  =  1,...,5 


with  the  set  of  i-truncated  action  profiles  for  player  i  denoted  by  A*'~*.  We  also  use  the  standard 
notation  a'  and  a~'  to  denote  the  action  profiles  for  player  i  and  the  action  profiles  of  all  other 
players  (similarly,  A^  and  A~*  are  used  for  the  sets  of  actions).  The  payoflF  functions  for  the 
players  depend  only  on  actions,  i.e.,  player  i's  payoff  is  given  by 


.i  ('^1 


(a\...,a"). 

We  assume  that  even  if  T  is  infinite,  the  game  admits  only  a  finite  set  of  payoff-relevant  classes 
of  histories.    We  also  define  the  restriction  of  the  payoff  function  u^  to  a  continuation  play 


(ai'*,...,a"'*)  asuMai'-*,...,a"'-*  :  a^i,...,af  :  a^'^+i, ...  ,a"'*+M  .  This  function  specifies 
the  utihty  of  player  i  when  the  action  profile  is  (a-^'~*, . . . ,  a"'~*)  up  to  and  including  time  t—1; 
then,  it  is  (aj,...,a")  at  time  t,  and  then  finally,  it  is  restricted  to  be  a^•^...,a"'*  from  t 
onwards.  Symmetrically,  this  payoff  function  can  also  be  read  as  the  utility  from  continuation 
action  profile  (a^'*"*"^, . . .  ja"'*"*"^)  given  that  up  to  time  t,  the  play  has  consisted  of  the  action 
profile  f  a^'~*, . . . , a"'~'    :    aj, . . .  ,a^  \. 

A  (possibly  mixed)  strategy  for  player  i  is 

where  A  {X)  denotes  the  set  of  probability  distributions  defined  over  the  set  X. 

Denote  the  set  of  strategies  for  player  i  by  S*.    A  ^-truncated  strategy  for  player  i  (corre- 
sponding to  strategy  a^)  specifies  plays  only  until  time  t  (including  time  t),  i.e.. 

The  set  of  truncated  strategies  is  denoted  by  S''~*.  A  i-continuation  strategy  for  player  i 
(corresponding  to  strategy  ct*)  specifies  plays  only  after  time  t  (including  time  t),  i.e., 

where  //*"■'  \  //*~^  is  the  set  of  histories  until  time  t  or  longer. 

With  a  slight  abuse  of  notation,  we  will  also  use  the  same  utility  function  defined  over 
strategies  (as  actions)  and  write 

U    UJ      ,G  \  il        ) 

to  denote  the  continuation  payoff  to  player  i  after  history  h*~^  when  he  uses  the 
continuation  strategy  ct*''  and  other  players  use  cr"''*.  We  also  use  the  notation 
u^  (  0-1'*, . . . ,  (t"'*  :  (7^'*+\  . . . ,  ct"'*+i  I  /i*-M  as  the  payoff  from  strategy  profile  (ct^-*,  . . . ,  ct"'*) 
at  time  t  restricted  to  the  continuation  strategy  profile  [a^'*'^^, . . . ,  cr"'*+^)  from  t  +  1  onwards, 
given  history  /i*"-^.  Similarly,  we  use  the  notation 

for  the  payoff  to  player  i  when  he  chooses  the  continuation  action  profile  a*'*  and  others  choose 
Q-i,t  given  history  h^~^. 

We  start  by  providing  the  standard  definitions  of  Nash  equilibria  and  subgame  perfect  Nash 
equilibria. 

6 


Definition  1  A  strategy  profile  ((T^,  . . . ,  ct")  is  a  Nash  Equilibrium  if 

u'  {a\  a-')  >  u'  {a\  a'')   for  all  a'  G  £*  and  for  alH  =  1, . . . ,  n. 
Definition  2  A  strategy  profile  (ct^,  ...,a^)  is  a  Subgame  Perfect  Nash  Equilibrium  if 

u'  (<T^'*,d-^'*  I  h'-^)  >  u'  (a''*,a-^'*  I  h*-^)  for  all  h*'^  G  H'-\ 

for  all  t,  for  all  a~'  G  E~*  and  for  alH  =  1, . . . ,  n. 

Sequentially  Weakly  Undominated  Equilibrium 

Towards  introducing  weakly  dominant  strategies,  let  us  take  a  small  digression  and  consider  a 
one  stage  game  with  actions  (a^,  ...,a"). 

Definition  3  We  say  that  (<t^,  . . .  ,(t")  is  a  weakly  undominated  equilibrium  if  for  each  i  = 
1, . . . ,  n,  for  any  cr*  G  S'  we  have 

u'{a\a-')  >u'{a\a-') 
(i.e.,  it  is  a  Nash  equilibrium)  and  for  each  i  =  1, . . . ,  n,  there  does  not  exist  cr'  G  E'  such  that 

u'{a\a-')  >u'{a\a-') 

for  all  (T~'  G  S~*  with  at  least  one  strict  inequality. 

Naturally,  such  equilibrium  always  exists  in  static  finite  games.  Let  us  next  extend  it  to  the 
general  T-stage  game  for  T  finite.  A  weakly  undominated  strategy  equilibrium  in  this  last  stage 
of  the  game  is  defined  analogously  to  Definition  3. 

Definition  4  Take  any  t  :  I  <  t  <  T  +  I.  Strategy  profile  (a\ . . .  ,(t")  is  a  h^~^ -sequentially 
weakly  undominated  equilibrium  if  (ct^,  . . .  ,(t")  is  a  /i*-sequentially  weakly  undominated  equi- 
librium for  any  /i*  that  may  occur  after  h*~^  and  for  each  i  =  1, . . . ,  n,  for  any  a*'*  G  E*'*, 

u   I  a  '  ,a    '     .CT,...,a  \  ti       j  ^  u   \  a  '  ,a    '     .    a  '      ,...,(T'        \  n       i      \\.) 

and  there  does  not  exist  ct*'*  G  £*'*  such  that 

u'  (b'\  a-*'*    ;    ai'*+\  . . . ,  a^'*+i  |  h'-^\  >  u'  U''\  a"*'*    \    a''*+\  . . . ,  a^'*+i  |  h^'A      (2) 

for  all  (T~*'*  G  E~*'*  with  at  least  one  strict  inequality. 


Definition  4  is  inductive,  but  this  induction  is  finite.  Indeed,  if  history  /i*""^  leads  to  a 
terminal  node,  then  the  first  condition  is  satisfied  automatically,  because  the  history  will  not  be 
recorded  any  further.  In  words,  we  first  hypothesize  that  there  exists  a  /i*-sequentially  weakly 
undominated  equilibrium,  impose  that  it  will  be  played  from  time  t  +  1  onwards  and  then  look 
for  a  weakly  dominant  strategy  profile  at  stage  t  of  the  game. 

Definition  5  Strategy  profile  ((j\  . . .  ,(t")  of  a  finite  extensive-form  game  is  a  Sequentially 
Weakly  Undominated  Equilibrium  (SWUE)  if  it  is  a  /i^-sequentially  weakly  undominated  equi- 
librium. 

Markov  Trembling-Hand  Perfect  Equilibria 

We  now  introduce  the  notion  of  Markov  Trembling-Hand  Perfect  Equilibria  (MTHPE).  For  this, 
we  need  to  define  a  Markovian  strategy  (Maskin  and  Tirole,  2001). 

Definition  6  A  continuation  strategy  ct*'*  is  Markovian  if 

for  aU  /i^"\  /i*~^  e  H*~'^  such  that  for  any  a*'*,  d*-*  e  A''*  and  any  a"'-*  G  A~^'* 

u'  (a''*,a-^'*  I  /i*-i)  >  u'  (a^'*,a-*'*  |  h'-^) 
implies  that 


U^\a-'''  I  h'-^)  >  u'  (a^'Sa"'-*  |  h''^ 


Definition  7  We  say  that  a  strategy  profile  (<j^,  ...,ct")  of  an  extensive-form  game  in  agent- 
strategic  form  is  Markov  Trembling-Hand  Perfect  Equilibrium  (MTHPE)  if  there  exists  a  se- 
quence of  totally  mixed  Markovian  strategy  profiles  { (a^  (m) , . . . ,  ct"  (m))  }^^^  (meaning  that 
continuation  strategy  a^''^  {m)  is  Markovian  for  all  z  =  l,...,n  and  all  m  £  N)  such  that 
((7^  (m) , . . . ,  ct"  (m))  -^  ((T^, . . . ,  (j")  as  m  ^  oo  and 

u*  {a\a~^  (m))  >  u*  (cr*, ct~'  (m))   for  all  a'  £  E\  for  all  m  e  N  and  for  alH  =  1, . . . , n. 

We  say  that  a  strategy  profile  (a^,  ...,0"")  of  an  extensive-form  game  in  strategic  form  is  Markov 
Trembling- Hand  Perfect  Equilibrium  (MTHPE)  if  it  is  MTHPE  in  the  corresponding  agent- 
strategic  form  game. 

Note  that  MTHPE  is  defined  directly  in  the  agent-strategic  form  in  order  to  avoid  standard 
problems  that  arise  when  trembling  hand  perfection  is  defined  on  the  strategic  form  (e.g.,  Selten, 
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1975,  Osborne  and  Rubinstein,  1994,  pp.  246-252).  Naturally,  MTHPE  is  a  stronger  concept 
than  THPE.^  Nevertheless,  we  establish  its  existence  under  general  assumptions  (Theorem  1). 
In  a  class  of  political  games,  which  we  refer  to  as  agenda-setting  games,  we  prove  that  a  pure- 
strategy  MTHPE  exists,  and  moreover  every  MTHPE  is  a  SWUE  (Theorem  2).  Nevertheless, 
these  two  equihbrium  concepts  do  not  always  coincide.  First,  a  MTHPE  always  exists,  while 
SWUE  may  not.  Second,  in  an  infinite  game,  there  may  exist  SWUEs  that  are  not  MTHPE. 

2.2     Agenda- Setting  Games 

Let  us  first  define  general  agenda-setting  games,  a  class  of  extensive  form  games  with  perfect 
information  that  includes  most  voting  games  as  a  special  case.^  In  Section  3,  we  establish  the 
existence  of  MTHPE  and  SWUE  for  these  games. 

To  define  a  general  agenda-setting  game,  we  first  need  a  definition  of  a  stage.  Inside  one  stage, 
information  sets  are  possible  as  the  definition  should  be  general  enough  to  nest  simultaneous 
voting. 

Definition  8  A  game  F  in  extensive  form  with  a  set  of  players  N  is  called  an  agenda- setting 
game  if  it  satisfies  the  following  conditions. 

1.  Game  F  consists  of  a  (possibly  infinite)  number  of  stages,  where  each  stage  0  is  either: 

(a)  a  "proposing  stage",  i.e.,  a  decision  node  and  a  number  of  possible  actions  of  a  single 
player  (possibly  Nature),  the  agenda-setter  at  stage  Q. 

(h)  a  "voting  stage",  i.e.,  a  connected  subgraph  of  F,  in  which  each  player  i  e  N  has  at 
most  one  decision  node  and  two  actions  at  this  node,  say  a^  (0)  and  a"  (0)  . 

2.  For  each  voting  stage  0,  there  are  only  two  equivalence  classes  of  continuation-payofi- 
identical  subgames  following  terminal  nodes  of  0  in  game  F,  say  y  (0)  and  n  (0). 

3.  In  each  voting  stage  0,  for  each  player  i  £  N  and  for  any  other  players'  actions  held  fixed, 
action  of  (0)  does  not  decrease  the  probability  of  moving  into  a  subgame  that  belongs  to 
the  equivalence  class  y  (0) . 


■*  Another  trembling  hand  refinement  used  in  the  literature,  truly  perfect  equilibrium,  is  stronger  than  our  notion 
of  MTHPE.  A  truly  perfect  equilibrium  requires  strategies  from  a  to  be  best  responses  to  all  fully  mixed  profiles 
in  some  neighborhood  of  a  rather  than  to  any  one  sequence  of  profiles  in  the  standard  notion  of  trembling-hand 
perfection  and  to  one  sequence  of  Markovian  profiles  in  the  case  of  MTHPE.  However,  this  equilibrium  concept 
fails  to  exist  in  many  games,  including  our  dynamic  game  of  coalition  formation  (except  in  some  special  cases). 

"The  study  of  agenda-setting  games  was  pioneered  by  Romer  and  Rosenthal  (1978)  and  Baron  and  Ferejohn 
(1989). 


4-  For  any  two  nodes  £,  and  ^'  of  game  F,  if  ^  and  ^  belong  to  one  information  set,  than  there 
exists  a  voting  stage  8'  of  game  F  that  contains  both  ^  and  ^' . 

This  definition  states  that  any  game  in  which  one  of  the  agents  makes  a  proposal  and  others 
vote  in  favor  or  against  this  proposal  is  an  agenda-setting  game  (and  imposes  the  intuitive 
restriction  that  more  "yes"  votes  do  not  make  a  proposal  less  likely  to  succeed).  Agenda-setting 
games  naturally  cover  all  dynamic  voting  games  in  which  at  some  stages  proposals  are  made, 
and  then  subsequently,  a  subset  of,  or  all,  players  vote  in  favor  or  against  these  proposals.  They 
also  include  several  dynamic  bargaining  games,  where  a  division  of  a  pie  is  offered  at  a  certain 
stage,  and  then  this  needs  to  be  accepted  by  the  other  participants  acting  simultaneously  or  in 
sequence,  as  special  cases. 

3     Characterization 

3.1      Existence  Results 

We  start  by  observing  that  MTHPE  exists  in  a  very  broad  class  of  games.  We  later  prove  that  it 
is,  nonetheless,  a  sufficiently  strong  equilibrium  refinement  to  rule  out  all  non-intuitive  equilibria 
in  agenda-setting  games. 

Theorem  1  Any  finite  or  infinite  extensive-form  game  with  a  finite  number  of  stages  has  a 
MTHPE  (possibly  in  mixed  strategies). 

Proof.  Suppose  first  that  T  is  finite.  Consider  a  perturbation  of  the  original  game  where  each 
player  i  £  N  is  restricted  to  play  each  action  at  each  stage  with  probability  t]\^>  t]  >  0,  where 
rj  is  sufficiently  small.  By  the  standard  fixed  point  theorem  argument,  this  perturbed  game 
has  a  Nash  equilibrium;  moreover,  the  fixed  point  theorem  applies  if  we  restrict  our  attention 
to  Markovian  strategies  only.  Therefore,  the  perturbed  game  has  a  Markov  Perfect  Equilibria 
(a^  (rj)  , . . .  ,a"  (77)).  Because  the  action  space  has  finite  dimensions  and  is  thus  compact,  we 
can  choose  a  sequence  77^,77^, .. .  which  converges  to  0  such  that  (a^  (rj^)  , . . .  ,a"  (t?'""))  has  a 
limit.  This  limit  would  be  a  trembling-hand  perfect  equihbrium  in  Markovian  strategies,  i.e.,  an 
MTHPE.   ■ 

If  T  is  infinite,  the  previous  reasoning  applies  straightforwardly  if  instead  of  "stages"  we  use 
"classes  of  stages  with  payoff-equivalent  histories" . 

Theorem  2        1.  Any  (finite)  agenda-setting  game  has  a  MTHPE  in  pure  strategies. 
2.  In  any  finite  game,  a  MTHPE  is  a  SWUE. 
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3.  Any  finite  agenda- setting  game  has  a  SWUE  in  pure  strategies. 

Proof.  (1)  We  proceed  by  induction  on  the  number  of  stages.  The  base  is  trivial;  indeed, 
consider  a  one-shot  agenda-setting  game.  If  this  stage  is  one-player  move,  then,  evidently,  the 
action  which  maximizes  his  utility  constitutes  a  pure-strategy  MTHPE,  since  it  is  Markovian 
and  trembling-hand  perfect.  If  the  single  stage  is  voting,  then  each  player  i  weakly  prefers  one 
of  the  outcomes  to  another.  It  is  straightforward  to  verify  that  voting  for  a  weakly  preferred 
outcome  is  an  MTHPE. 

We  next  proceed  with  the  induction  step.  Suppose  that  we  have  proved  the  existence  of 
pure-strategy  MTHPE  in  all  agenda-setting  games  with  number  of  stages  less  than  T;  take  an 
agenda-setting  game  with  T  stages.  Consider  its  first  stage.  We  start  with  the  case  where 
there  is  one  player  i  making  a  choice  between  k  actions  ai, . . . ,  a^.  In  each  of  k  corresponding 
subgames  there  exists  (by  the  induction  hypothesis)  a  pure  strategy  MTHPE;  we  can  choose  the 
same  MTHPE  for  isomorphic  subgames.  Therefore,  there  exist  sequences  of  strategy  profiles 
ctJ,  a^, . . .  for  each  j  G  {1, . . . ,  /c}  which  converge  to  a  pure  strategy  MTHPE  for  each  such  j  and 
which  are  Nash  equilibria  in  the  corresponding  T  —  1-stage  games.  Moreover,  we  can  require 
that  if  two  subgames  (for  j  =  j\,J2)  are  isomorphic,  then  ct"  =  cr",  for  any  n.  Now,  consider 
player  i  at  first  stage.  For  any  n,  there  is  j„  such  that  Cj^  is  weakly  better  than  other  actions. 
Since  there  are  a  finite  number  of  actions,  there  is  action  aj  that  is  weakly  better  than  other 
actions  for  infinitely  many  values  of  n.  It  is  now  straightforward  to  prove  that  aj,  along  with 
the  chosen  MTHPE,  forms  a  pure  strategy  MTHPE  of  the  whole  game. 

Now  consider  the  case  where  the  initial  stage  is  a  voting  stage.  Then  there  are,  essentially, 
(at  most)  two  subgames;  take  a  pure-strategy  MTHPE  in  each  (if  they  are  isomorphic,  take 
the  same  MTHPE),  and  we  can  similarly  construct  two  sequences  of  strategy  profiles  cr^,  cr^, . . . 
and  (j\,a\,...  Each  player  i  has  two  actions,  y  and  n,  in  stage  1.  Consider  one  of  the  players 
(i)  and  take  any  m.  If  in  the  subgames  strategies  a"^  and  a^  are  played,  respectively,  then 
player  i  weakly  prefers  to  choose  one  of  the  actions,  y  or  n,  to  another  one.  There  is  an  action 
a  [i)  which  is  weakly  preferred  for  infinite  number  of  m's.  Now  consider  two  subsequences  of 
sequences  ct^,  cr^, . . .  and  u^,  ct^,  . . .  which  are  formed  by  values  of  m  for  which  a  {i)  is  weakly 
better  for  player  i.  Take  another  player,  j,  and  repeat  the  procedure;  then  we  will  get  action 
a  (j)  and  two  subsequences  of  the  previous  subsequences.  Since  the  number  of  players  is  finite, 
we  can  proceed  in  this  way,  and  for  any  player  i  find  action  a{i).  It  is  now  evident  that  the 
MTHPEs  chosen  for  the  stages  starting  from  the  second  one  and  actions  a  (z)  for  each  player  i 
at  the  first  stage  form  a  pure  strategy  MTHPE. 
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(2)  Take  any  strategy  profile  a  that  forms  a  MTHPE.  This  is  proved  by  backward  induction 
on  the  number  of  stages  in  the  game.  Suppose  that  the  Theorem  has  been  proved  for  games 
with  q'  <  q  stages.  Consider  an  agenda-setting  game  with  q  stages  and  take  any  MTHPE  in 
it.  By  induction,  this  MTHPE,  when  truncated  to  any  of  the  game's  proper  subgames,  forms  a 
SWUE.  Consider  its  first  stage. 

Suppose  that  only  one  player  i  moves  at  this  stage;  denote  his  expected  utility  (in  this 
MTHPE)  from  taking  action  a  at  first  stage  by  u°.  If  action  a*  is  an  action  played  with  a 
non-zero  probability  in  equilibrium  then  u^'  >  uf  for  any  other  feasible  action  a  (otherwise 
there  would  exist  a  payoff-improving  deviation).  Hence,  all  actions  played  in  a  MTHPE  with  a 
non-zero  probability  yield  the  same  expected  utility  for  player  i,  and  this  utility  is  the  maximum 
possible  over  the  set  of  feasible  actions.  Hence,  this  MTHPE  is  a  SWUE. 

Now  consider  the  other  situation  where  the  first  stage  is  a  voting  stage.  Consider  a  profile  a' 
consisting  of  fully  mixed  strategies  and  suppose  that  it  is  ?7-close  to  a  for  a  small  ry.  Depending 
on  how  other  players  vote,  three  mutually  exclusive  situations  are  possible:  proposal  is  accepted 
regardless  of  how  player  i  votes,  it  is  rejected  regardless  of  how  he  votes,  and  player  i  is  pivotal; 
let  /x"*",  iJ,~ ,  and  ^^  be  the  respective  probabilities  of  these  events.  By  definition,  ^'^+^~+fi'P  =  1, 
and  by  assumption  fi^  >  0.  Voting  for  the  proposal  yields  (/i"*"  -I-  n^)  u^'  +  A*~'"i~'  in  expectation, 
voting  against  it  yields  fx'^uf  +  {fx~  +  pf)  u~'  where  u^'  and  u~'  are  i's  utilities  from  acceptance 
and  rejection  of  the  proposal  if  profile  a'  is  played.  Thus,  if  u^'  >  u~',  then  player  i's  sole  best 
response  is  voting  for  the  proposal,  and  if  u'^'  <  u~'  it  is  voting  against  it.  If  rj  is  sufficiently 
small  then  u^  >  u~  implies  uf  >  u~' ,  and  thus  by  definition  of  MTHPE  player  i  must  support 
the  proposal  in  equilibrium  with  probability  one.  Similar  reasoning  applies  to  the  case  uf  <  u~ . 

Now  take  any  player  i  who  participates  in  voting.  If  uf  >  u~ ,  then  he  votes  for  the  proposal 
in  this  MTHPE.  This  is  a  weakly  dominant  strategy  for  him  (given  continuation  strategies  of 
himself  and  other  players) .  Similarly,  if  uf  <  u~ ,  then  the  strategy  he  plays  in  this  MTHPE  is 
weakly  dominant.  If,  u^  =  u~  or  the  player  is  never  pivotal,  then  any  strategy  is  weakly  undom- 
inated.  Therefore,  for  any  player,  the  strategy  he  plays  in  this  MTHPE  is  weakly  undominated, 
and  thus  this  MTHPE  is  a  SWUE.  This  completes  the  induction  step. 

(3)  This  follows  from  (1)  and  (2)  of  this  Theorem.    ■ 

Existence  of  an  MTHPE  equilibrium  in  pure  strategies  is  at  first  a  surprising  result  and 
will  be  exploited  below  for  a  full  characterization  of  these  equilibria  in  agenda-setting  games 
(existence  of  an  SWUE  in  pure  strategies,  which  can  be  computed  by  a  backward-induction 
procedure,  is  obvious). 
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(0,0)  (1,1)  (1,1) 

Figure  3:  A  Game  With  SWUE  which  is  not  MTHPE 

3.2     Relationship  Between  MTHPE  and  SWUE 

However,  SWUE  and  MTHPE  concepts  are  not  equivalent,  and  neither  is  in  general  stronger 
than  the  other  (though  see  Theorem  4  below). 

Example  3  There  exists  an  agenda-setting  game  where  an  SWUE  is  not  an  MTHPE.  Consider 
a  game  of  two  players  with  extensive  form  depicted  on  Figure  3.  This  is  an  agenda-setting  game, 
because  at  each  stage  only  one  player  has  a  (non-trivial)  move.  It  game  has  exactly  one  MTHPE 
{R,r).  However,  there  are  two  SWUEs:  {R,r)  and  {L,r).  The  latter  is  not  MTHPE,  because  if 
there  is  a  non-zero  chance  that  player  2  will  play  I,  player  1  is  better  off  choosing  R. 

As  an  alternative  to  the  concept  of  SWUE,  one  might  consider  Sequentially  Weakly  Dominant 
Equilibria,  defined  as  follows: 

Definition  9  Take  any  t:l<t<T+l.  Strategy  profile  (ct^,  . . .  ,ct")  is  a  h^~^  -sequentially 
weakly  dominant  equilibrium  if  ((t\  . . .  ,(t")  is  a  h* -sequentially  weakly  dominant  equilibrium  for 
any  h}  that  may  occur  after  /i*~^  and 

a '  ,a  :    a         , . . .  ,a  \  ti       I     >     u   [  a    ,a         .    a  '      ,...,(7'        \  n 

for  all  CT*'*     G     i;*'*,  for  all  cr'''^  G  S"^'*,  and  for  all  i  =  I,. . .  ,N. 

(Here,  we  use  the  convention  that  any  profile  is  h^ -sequentially  weakly  dominant  equilibrium.) 
Strategy  profile  (ct^,  . . .  ,17")  is  a  Sequentially  Weakly  Dominant  Equilibrium  (SWDE)  if  it  is  a 
h^ -sequentially  weakly  dominant  equilibrium. 

Note  that  we  do  not  require  that  the  inequality  holds  strictly  for  at  least  one  (7''*,cr~*'*  in 
order  to  ensure  existence  in  situations  where  one  player  is  indifferent  between  different  outcomes. 
Clearly,  SWDE  is  a  much  stronger  equilibrium  concept  than  SWUE.  Nevertheless,  it  can  be 
shown  that  for  agenda-setting  games  the  notions  of  SWDE  and  SWUE  coincide,  though  naturally 
for  general  finite  games  SWDE  is  a  "strictly"  stronger  concept  and  may  fail  to  exist.  More 
precisely,  we  can  establish  the  following  result. 
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Theorem  3        1.   In  any  finite  agenda- setting  game,  a  SWUE  is  a  SWDE  (and  thus  a  MTHPE 
is  a  SWDE).  In  particular,  such  games  have  SWDE  in  pure  strategies. 

2.  In  any  finite  game,  a  SWDE  is  a  SWUE  (so  for  agenda- setting  games  these  notions  coin- 
cide). 

3.  There  exists  a  finite  game  where  a  MTHPE  is  not  a  SWDE. 

4-   There  exists  an  agenda-setting  game  where  a  SWDE  is  not  a  MTHPE. 

Proof.  (1)  This  is  easily  proven  by  induction.  Take  any  SWUE  a.  The  base  is  trivial  (essen- 
tially, it  may  be  proved  with  exactly  the  same  words  as  the  induction  step).  Step:  take  any 
stage;  if  only  a  single  player  i  acts,  then  both  SWUE  and  SWDE  allow  for  any  action  that  is  a 
best  response  of  player  i  given  that  a  is  played  from  that  stage  on.  Suppose  the  stage  involves 
voting;  take  some  player  i.  If  he  is  pivotal  with  probability  zero,  then  any  strategy  would  con- 
stitute a  SWDE,  in  particular,  the  one  he  plays  in  profile  a.  Now  assume  that  he  is  pivotal  with 
a  positive  probability.  Consider  first  the  case  where  i  plays  a  pure  strategy;  then  without  loss 
of  generality  assume  that  SWUE  prescribes  that  player  i  votes  "for"  the  proposal.  Denotes  his 
continuation  payoffs  if  the  proposal  is  accepted  and  is  rejected  by  u'^  and  u'^ ,  respectively,  then 
in  SWUE  <T,  uf  >  u~  must  hold  (according  to  Definition  4;  otherwise  voting  "for"  would  be 
a  dominated  strategy  at  this  stage).  But  in  that  case,  voting  "for"  is  also  a  weakly  dominant 
strategy:  indeed,  voting  "against"  is  the  only  alternative  action,  and  hence  u'l  >  uf  for  any 
action  a.  Finally,  if  a  prescribes  a  mixed  strategy  (and  i  is  pivotal  with  a  positive  probability), 
if  must  be  that  u^  =  u~ .  But  then  again,  any  player  i's  strategy  at  this  stage  forms  a  SWDE 
after  the  corresponding  history.  This  proves  the  induction  step. 

(2)  Again,  this  is  proved  by  induction.  Take  any  finite  game  and  SWDE  a.  The  base  is  again 
trivial.  Step:  take  any  player  i  and  suppose  that  he  plays  strategy  ct*'*  G  !]*•'.  This  implies,  by 
definition,  that 

Ui{a''',a-''')  >u^{a'^',a-''')  (3) 

(we  drop  conditioning  on  past  and  future  actions  for  brevity)  for  any  a^'^  and  u"*'*.  This 
immediately  implies  that  condition  (1)  holds  for  any  a''*  G  S*'*.  Now  assume,  to  obtain  a 
contradiction,  that  there  is  ct''*  G  S'''  such  that  for  at  least  one  combination  of  other  players' 
strategies  ct~*'*  G  S"'-*  we  have 

Ui  [cr  '  ,a    '  )  >  Ui  {^a  '  ,  a      ) 

(recall  that  Definition  4  requires  that  one  of  the  inequalities  (2)  be  strict).  However,  this  last 
condition  violates  (3).  This  contradiction  completes  the  proof. 
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(3)  Consider  a  one-stage  game  with  two  players  making  simultaneous  moves  with  payoff 

matrix 

I  r 

L     (1,1)     (0,0)  . 
R    (0,0)     (1,1) 

This  game  does  not  have  SWDE,  because  it  is  one-stage  and  in  that  only  stage  neither  of  the 

players  has  a  weakly  dominant  strategy.  It  is  straightforward  to  check,  however,  that  both  (X,  I) 

and  {R,  r)  are  MTHPEs  of  this  game. 

(4)  This  follows  from  Example  3.  Indeed,  in  this  example  the  SWUE  is  also  a  MTHPE,  as 
any  SWUE  is  SWDE  in  an  agenda-setting  game.  Hence,  there  is  an  agenda-setting  game  where 
SWDE  is  not  MTHPE.   ■ 

3.3     MTHPE  in  Agenda-Setting  Games 

In  this  subsection,  we  provide  a  characterization  of  the  structure  of  MTHPE  in  agenda-setting 
games.  We  define  the  notion  of  a  regularization  of  an  agenda-setting  game.  A  regularization 
of  an  agenda  setting  game  transforms  it  into  an  alternative  game  in  which  at  each  stage  only 
one  player  takes  an  action,  i.e.,  stages  where  there  are  simultaneous  moves  are  replaced  by  a 
sequence  of  stages  corresponding  to  each  one  of  these  moves. 

Definition  10  Suppose  that  T  is  a  (finite)  agenda-setting  game.  We  call  game  F'  a  regulariza- 
tion of  game  V  if  it  is  obtained  from  T  by  substituting  every  stage  where  more  than  one  player 
moves  by  a  sequence  of  stages  such  that  each  stage  only  one  player  takes  an  action.  If  there  is 
voting  among  players  X  C  N  at  stage  ^,  then  ^  is  substituted  by  \X\  stages  in  which  players 
i  &  X  move  sequentially.  Player  i  has  the  same  two  actions  that  he  had  at  stage  ^,  and  the  game 
proceeds  with  subgame  y  (^)  after  profile  {at  (Oliex  ^^  played  if  and  only  if  the  original  game  T 
proceeds  with  y  {£,)  after  {cj  (C)}iex  ^^  played  at  stage  £,. 

Clearly  there  are  several  regularizations  of  an  agenda-setting  game  that  has  simultaneous 
actions  by  two  or  more  players  at  the  same  stage  (and  the  regularization  of  an  agenda-second 
game  is  itself  an  agenda-setting  game,  with  the  unique  regularization  that  is  itself).  Crucially, 
however,  each  of  these  regularizations  has  the  same  set  of  terminal  payoffs.  We  next  show  that 
different  regularizations  will  also  lead  to  the  same  set  of  equilibrium  payoffs  to  all  players. 

To  establish  this  result,  first  note  that  any  strategy  profile  a  of  game  T  can  be  naturally 
mapped  to  a  strategy  profile  a'  of  a  regularization  F',  so  that  players  play  the  same  actions 
as  in  profile  a.  With  a  slight  abuse  of  notation,  we  say  that  u  is  a  strategy  profile  of  game 
F'.    Notice  that  the  converse  of  the  statement  is  not  true,  since  a'  may  involve  a  pattern  of 
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actions  that  cannot  be  captured  by  cr,  for  example,  when  later  actions  depend  on  earlier  actions. 
Nevertheless,  the  next  theorem  shows  a  strong  equivalence  between  the  set  of  MTHPE  of  an 
agenda-second  game  and  the  set  of  MTHPE  (and  SPE)  of  any  regularization  of  the  same  game. 

Theorem  4  Suppose  T  is  a  finite  agenda-setting  game. 

1.  If  a  is  a  MTHPE  of  game  F,  then  it  is  an  MTHPE  (and  thus  an  SWUE  and  a  SPE)  m 
any  regularization  T' . 

2.  Conversely,  if  a'  is  a  MTHPE  in  regularization  T' ,  then  there  exists  an  MTHPE  a  in  T 
in  which  all  players  obtain  the  same  payoffs  as  in  a' . 

Proof.  (1)  Take  MTHPE  a  of  game  F.  Denote  the  corresponding  strategy  profile  in  game 
F'  by  a'  to  avoid  confusion.  Consider  a  sequence  of  fully  mixed  profiles  (g^  {r) ,. ..  ,a^  (r)) 
which  converge  to  the  equilibrium  profile  (cr^,. . .  ,(7")  as  r  — >  oo  and  such  that  the  ct*'*  is  a 
best  response  to  a  (r)  for  every  player  i  at  every  stage  or  node  ^  (this  exists  by  the  definition 
MTHPE).  Since  the  game  is  finite,  we  may,  without  loss  of  generahty,  assume  that  at  every 
voting  stage  every  player  that  participates  in  the  voting  has  the  same  preferences  over  the  two 
alternatives  y  and  n  for  all  r's  in  the  sequence  (i.e.,  we  can  always  take  such  subsequence). 
This  implies  that  at  voting  stage,  each  player  either  strictly  prefers  y,  or  strictly  prefers  n,  or  is 
indifferent  between  the  two  for  all  r. 

Consider  the  corresponding  profiles  {a'^  (r) , . . .  ,(t'"  (r))  of  the  regularized  game  F';  obvi- 
ously, they  are  fully  mixed  and  converge  to  a'  as  r  — »•  oo.  Let  us  verify  that  a'  is  a  best  response 
to  (cr'^  (r) , . . . ,  a'^  (r))  for  every  player  i  at  any  node  ^  and  for  any  r.  If  node  £,  corresponds  to 
a  single  action  by  some  player  in  game  F,  then  it  is  obvious,  since  the  payoffs  of  all  actions  are 
the  same,  as  the  continuations  of  profiles  (ct'^  (r) , . . . ,  cr'"  (r))  and  (ct^  (r) , . . . ,  ct"  (r))  have  the 
same  distribution  over  the  payoffs  at  terminal  nodes.  Consider  the  case  where  ^  corresponded 
to  a  voting  in  game  F.  If  i  were  indifferent  about  the  outcomes  of  the  voting  in  F  (provided 
that  (cr^  (r) , . . .  ,(t"  (r))  is  played  after  that  voting),  then  he  is  indiflferent  in  the  game  F'  as 
well,  for  the  same  reason  as  before.  If  he  strictly  preferred  one  of  the  outcomes  (say,  y(C))i 
then  he  must  have  picked  it  in  game  F  (since  he  was  pivotal  with  a  positive  probability  since 
strategies  were  fully  mixed).  In  the  game  F',  if  profile  (tr'^  (r) , . . . ,  a'"  (r))  is  played,  his  choice 
of  action  af  (^)  cannot  decrease  the  likelihood  that  the  corresponding  subgame  is  reached  after 
this  voting,  because  other  players'  strategies  in  this  (now  sequential)  voting  do  not  depend  on 
his  action.  Consequently,  choosing  af  (^)  is  still  a  best  response  for  him.  The  case  where  player 
i  strictly  preferred  outcome  n  {£,)  is  treated  similarly.  This  establishes  that  profile  a'  consists  of 
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best  responses  to  (cr'^  (r) , . . . ,  cr'^  (r))  for  all  players,  at  all  stages,  and  for  all  r's,  i.e.,  a'  is  a 
MTHPE. 

(2)  Take  any  MTHPE  a'  in  regularization  F'.  Consider  a  sequence  of  fully  mixed  profiles 
(^a'^  (r) , . . . ,  (t'"  (r))  which  converge  to  the  equilibrium  profile  (<7'\  . . . ,  a'")  as  r  ^  oo  and  such 
that  the  ct"'*  is  a  best  response  to  a'  (r)  for  every  player  i  at  every  stage  or  node  ^.  As  in 
part  1 ,  we  may,  without  loss  of  generality,  assume  that  at  every  voting  stage  every  player  who 
participates  in  the  voting  has  the  same  preferences  over  the  two  alternatives  y  and  n  for  all  r's 
in  the  sequence  (note  that  these  preferences  are  well-defined,  since  a'  must  consist  of  Markovian 
strategies). 

For  any  voting  node  ^  in  game  F,  take  the  probability  p  {£,)  that  the  proposal  will  be  accepted 
in  the  corresponding  voting  in  game  F'  under  profile  a'.  The  probability  is  taken  conditional  on 
reaching  this  node;  note  that  this  probability  is  well  defined  in  the  sense  that  it  is  the  same  for 
all  votings  in  F'  that  may  correspond  to  node  ^  in  F  since  a'  consists  of  Markovian  strategies. 
One  can  easily  check  the  following  statements  about  votings  in  F'.  If  p(C)  <  1,  then  the  set  of 
players  who  strictly  prefer  y  (^)  to  n  (^)  (for  all  r's)  is  not  sufficient  for  y  (^)  to  be  reached  even 
if  they  all  voted  af  (^).  Similarly,  if  p  (^)  >  0,  then  the  set  of  players  who  strictly  prefer  n  (^)  to 
y  {^)  is  not  sufficient  for  n  (^)  to  be  reached.  Moreover,  if  p(0  ^  (Oil)>  then  there  is  a  player 
who  is  indifferent  between  y  (^)  and  n  {^)  subgames  for  all  r. 

Let  us  now  define  strategy  profile  a  as  follows.  At  a  node  where  player  i  makes  a  single 
decision,  let  him  choose  the  same  action  as  in  corresponding  nodes  of  game  F'  (there,  he  chooses 
the  same  action  as  a'  is  Markovian).  At  a  voting  node  i^  with  p  (0  =  0,  let  all  players  who 
prefer  n  to  y  or  were  indifferent  (for  all  r  above)  choose  a"  (^),  and  let  the  rest  choose  of  (^).  If 
p  (^)  =  1,  let  all  players  who  prefer  y  to  n  or  are  indifferent  choose  a^  {^)  and  let  the  rest  choose 
af  (^).  Finally,  if  p{^)  G  (0, 1),  then  let  all  players  who  strictly  preferred  y  choose  af  (^),  all 
players  who  strictly  preferred  n  choose  a"  (^),  and  let  those  who  were  indifferent  mix  in  such  a 
way  that  the  probability  of  acceptance  is  p  {£,)  (this  is  possible,  since  neither  of  the  two  groups 
with  strict  preferences  is  sufficient  to  enforce  either  of  the  decisions). 

Let  us  verify  that  ct  is  a  MTHPE.  Observe  that  for  any  r  we  can  take  a  fully  mixed  profile 
(fj^  [r) ,. . .  ,a^  (r))  which  converges  to  a  and  that  for  every  subgame  in  F  have  the  same  payoffs 
for  all  players  as  profiles  (a'^  (r) , . . . ,  cr'"  (r))  chosen  above.  Then  at  each  node  of  F  players  play 
best  responses,  due  to  the  way  we  defined  profile  a.  This  completes  the  proof.    ■ 

This  result  establishes  the  equivalence  of  agenda-setting  games  with  simultaneous  and  se- 
quential votings  once  a  proper  equilibrium  concept  (MTHPE)  is  used.  More  importantly,  it 
also  highlights  the  importance  of  MTHPE  as  an  equilibrium  refinement.  Example  6  shows  that 
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part  (1)  of  Theorem  4  would  not  hold  if  we  considered  THPE  instead  of  MTHPE.  Example  5 
in  the  Appendix  shows,  among  other  things,  that  it  would  also  not  hold  if  we  considered  se- 
quential elimination  of  weakly-dominated  strategies  (and  in  particular,  with  this  notion  certain 
non-intuitive  equilibria  still  survive).  Thus,  the  equivalence  result  for  SWUE  does  not  hold. 

Finally,  the  next  theorem  estabhshes  a  "generic  uniqueness"  of  MTHPE  in  agenda-setting 
games.  Example  6  in  the  Appendix  again  shows  that  it  would  not  hold  for  THPE. 

Theorem  5  Suppose  agenda-setting  game  T  is  such. that  for  any  two  terminal  nodes,  if  one 
player  obtains  the  same  payoffs  in  these  two  nodes,  then  each  player  obtains  the  same  payoffs 
in  both  nodes.  Then  in  any  two  MTHPEs  ui  and  (72,  all  players  obtain  the  same  payoffs,  and 
these  payoffs  are  the  same  as  those  in  any  SPE  of  any  regularization  ofT'. 

Proof.  We  prove  the  following  statement  by  induction  by  the  number  of  stages:  (a)  for  any 
two  MTHPEs  (Ti  and  fj2  all  players  get  the  same  payoffs,  and  (b)  if  at  some  MTHPE  a  players 
get  a  certain  expected  payoff  vector,  then  there  exists  a  terminal  node  where  players  get  these 
payoffs.  Let  us  prove  the  stage  (the  base  is  trivial).  If  the  first  node  of  the  game  involves  player 
i  making  a  decision,  then  of  all  available  actions  he  will  choose  one  which  maximizes  his  utility. 
If  there  are  several  such  actions  then,  by  induction,  the  payoff  vectors  of  the  corresponding 
subgames  are  realized  at  some  terminal  nodes  of  the  subgames.  Since  player  i  is  indifferent,  he 
gets  the  same  payoff,  and  by  assumption  all  players  get  the  same  payoff.  This  proves  part  (a) 
of  the  statement;  part  (b)  is  trivial.  Consider  the  case  where  the  first  stage  involves  voting.  By 
induction,  either  all  players  are  indifferent  between  the  alternatives  y  and  n,  or  all  players  have 
a  strict  preference.  In  the  first  case,  all  MTHPEs  involve  the  same  payoffs  for  trivial  reasons, 
while  in  the  second,  the  MTHPE  strategies  of  players  are  unique,  and  hence  the  payoffs  are  the 
same.  Again,  part  (b)  is  trivial.  This  proves  the  induction  step. 

To  complete  the  proof,  we  need  to  show  that  these  payoffs  coincide  with  the  payoffs  in  any 
SPE  of  any  regularization.  By  Theorem  4,  all  players  get  these  payoffs  in  any  MTHPE  of  the 
game  (and  thus  in  some  SPE  of  the  game).  It  remains  to  prove  that  in  a  game  where  players 
move  one  at  a  time  and  where  in  any  two  terminal  nodes  either  all  players  get  the  same  payoffs 
or  all  players  get  different  payoffs,  the  payoffs  of  all  players  in  any  two  SPEs  are  the  same.  This 
proof  may  be  done  by  induction,  similarly  to  the  one  in  the  beginning  of  this  proof.    ■ 

4     Application:  the  Elimination  Game 

To  demonstrate  the  usefulness  of  the  MTHPE  and  SWUE  concepts,  we  now  use  a  version  of  the 
endogenous  club  formation  game  (or  "pohtical  elimination"  game)  studied  in  Acemoglu,  Egorov, 
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and  Sonin  (2008).  This  is  an  infinite-horizon  dynamic  game  with  the  following  structure:  there 
is  an  initial  collection  of  individuals,  N  =  {1, . . .  ,n},  with  each  individual  i  €  N  endowed  with 
political  power  7^  >  0  (for  example,  representing  individual  z's  "guns").  At  each  date  t  >  1, 
there  exists  a  ruling  coalition  Xt  C  A^,  and  members  of  this  ruling  coalition  may  vote  to  exclude 
some  of  its  members,  thus  determining  a  new  ruling  coalition  Xt+i  C  Xt-  At  time  t  =  0,  we 
have  Xq  =  N.  The  total  power  of  any  coalition  X  C  A^  is 

We  assume  that  powers  {ijjjgyv  ^^^  generic  in  the  sense  that  if  X,Y  c  A^  and  X  ^  Y ,  then 

Ix  +  7y- 

In  any  period  i  >  1,  the  voting  proceeds  as  follows.  An  agenda-setter,  which  is  determined  by 
Nature  at  random  (or  through  any  fixed  protocol)  proposes  an  alternative  coalition  At^i  which  is  a 
subcoalition  of  the  current  coalition.  All  individuals  who  are  entitled  to  do  so  vote  for  or  against 
At^i-  Alternative  An  becomes  the  next  ruling  coalition,  i.e.,  Xt^\  =  Aj^j,  if  and  only  if  it  receives 
an  absolute  majority  of  the  available  ''weighted  votes,^^  where  votes  are  weighted  by  the  power 
of  each  member  of  the  junta,  so  that  an  individual  with  a  greater  7j  has  proportionately  more 
votes.  If  the  proposal  At^t  does  not  receive  an  absolute  majority,  then  the  next  agenda-setter 
nominates  a  proposal  and  so  on.  In  case  no  proposal  is  accepted,  Xt+i  =  Xt- 

The  preferences  (for  each  i  £  N)  consist  of  two  parts.  The  first  is  utility  from  power, 

oo 

[/+  =  (l-/3)]B,^/3--*;^I,ex.. 

T=t  ^^t 

Here  /3  G  (0, 1)  is  the  discount  factor  common  across  all  individuals,  li^Xr  is  the  indicator 
function  for  individual  j  being  a  member  of  the  ruling  coalition  at  time  t,  and  Et  denotes 
expectations  at  time  t.  The  term  7j/7xt  represents  the  power  of  the  individual  relative  to  other 
ruling  coalition  members.  It  can  be  motivated  from  the  division  of  a  unit  size  pie  among  the 
coalition  members  in  proportion  to  their  power.  The  important  implication  of  this  functional 
form  is  that  each  player  obtains  greater  utility  when  the  power  of  the  ruling  coalition  is  smaller. 
Therefore,  each  prefers  to  be  a  member  of  a  smaller  ruling  coalition.^ 

Formally,  the  extensive  form  of  the  game  is  as  follows.  We  assume  that  an  agenda-setter  is 
picked  randomly  by  nature;  yet  all  the  results  go  through  for  any  fixed  protocol  of  agenda-setters. 

1.  At  each  stage,  j  =  0, 1, . . .,  the  game  starts  with  an  intermediary  coalitions  by  Nj  C  N 
(with  No  =  N). 


^  These  payoffs  are  a  special  case  of  the  more  general  payoffs  introduced  in  Acemoglu,  Egorov  and  Sonin  (2008) 
and  are  adopted  to  simplify  the  notation  here  (without  affecting  any  of  the  results). 
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2.  Nature  randomly  picks  agenda  setter  ij^g  e  Nj  ior  q  =  1  (i.e.,  a  member  of  the  coalition 

3.  Agenda  setter  ij^q  proposes  a  coalition  Xj^q  G  P  (Nj). 

4.  All  players  in  Xj^q  simultaneously  vote  over  this  proposal;  let  Vj^q  {i,  Xo,q)  G  {y,  n}  be  the 
vote  of  player  i  G  Xj^q.  Let  Yes  {Xj^q}  be  the  subset  of  Xj^g  voting  in  favor  of  this  proposal 
(i.e.  YesjXj,,}  =  {z  G  X^,,  |  Vj^q  {i,Xo,q)  =  y}).  Then,  if 

i£Yes{A'j,,}  i&Nj 

i.e.,  if  Xj^q  has  the  absolute  majority  of  the  weighted  votes  within  Nj,  then  the  game 
proceeds  to  step  5;  otherwise  the  game  proceeds  to  step  6. 

5.  If  Xj^g  =  Nj,  then  we  proceed  to  step  7  and  the  game  ends.  Otherwise  players  from 
Nj  \  Xj^q  are  eliminated,  players  from  Xj^q  add  — e  to  their  payoff,  and  the  game  proceeds 
to  step  1  with  Nj+i  =  Xj^q  (and  j  increases  by  1). 

6.  li  q  <  \Nj\,  then  next  agenda  setter  ij,q+i  G  Nj  is  randomly  picked  by  nature  such  that 
b,9+i  7^  ^j,r  ioT  1  <  r  <  q  (i.e.,  it  is  picked  among  those  who  have  not  made  a  proposal  at 
stage  j)  and  the  game  proceeds  to  step  3  (with  q  increased  by  1).  Otherwise,  we  proceed 
to  step  7. 

7.  Nj  is  becomes  the  ruling  coalition  of  this  terminal  node,  and  each  player  i  G  Nj  adds 
Wi  {Nj)  to  his  payoff. 

Coalitions  that  emerge  during  the  game  form  a  sequence  Nq  D  Ni  D  . . .  D  Nj  where  j  is 
the  number  of  coalitions  (except  initial  one)  that  emerges  during  the  game.  Whenever  there  is 
a  non-trivial  transition,  each  player  incurs  a  disutility  cost  equal  to  e  >  0  (e  can  be  arbitrarily 
small).  Summing  over  the  payoffs  at  each  node,  the  payoff  of  each  player  i  is  given  by 

•         Ui  =  Wi  (Nj)  -eY,  In,  (i) ,  (4) 

i<j<J 

where  Ix  (•)  is  the  indicator  (characteristic)  function  of  set  X.  This  payoff  function  captures  the 
fact  that  individuals'  overall  utility  in  the  game  is  related  to  their  share  Wi  and  to  the  number 
of  rounds  of  elimination  in  which  the  individual  is  involved  in  (the  second  term  in  (4)).  The 
number  of  players  ehminated  equals  |A'^|  -  |A'j|,  and  there  are  totally  J  rounds  of  elimination. 
With  a  slight  abuse  of  terminology,  we  refer  to  j  above  as  "the  stage  of  voting." 
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The  next  two  examples  show  that,  as  in  Examples  1  and  2  in  the  Introduction,  in  this 
game  there  are  also  several  "non-intuitive"  SPEs,  and  in  fact,  there  are  several  "non-intuitive" 
Markovian  SPEs. 

Example  4  Consider  N  =  {A,B,C,D},  with  7^  =  3,7^  =  4,  7^  =  5  and  7^,  =  10,  and 
suppose  that  a  =  1/2.  It  can  be  verified  that  there  is  a  MTHPE  where  A,  B,  or  C  proposes  the 
coalition  {A,B,C},  and  following  this  proposal.  A,  B  and  C  vote  for  in  favor  of  this  coalition 
(and  if  D  is  the  first  to  make  a  proposal,  he  proposes  {A,B,C,D}  which  is  rejected  by  the 
rest).  The  fact  that  {A^B,C}  should  be  the  ultimate  ruling  coalition  is  intuitive:  any  two- 
person  subset  of  it  is  "unstable"  and  would  lead  to  the  elimination  of  one  of  its  members.  And 
clearly.  A,  B,  and  C  would  prefer  {A,  B,  C}  to  {A,  B,  C,  D}.  Nevertheless,  there  exists  another 
SPE,  where  all  players  make  proposals  according  to  a  pre-fixed  protocol  and  everybody  votes 
against  any  proposal.  This  is  indeed  an  SPE,  because  no  deviation  by  a  single  player  can  lead 
to  acceptance  of  any  proposal.  Consequently,  both  {A,  B,  C}  and  {A,  B,  C,  D}  can  emerge  as 
outcomes  in  a  SPE,  even  though  the  former  is  not  a  reasonable  outcome,  since  A,  B,  and  C  have 
enough  votes  to  eliminate  D.  We  can  also  note  that  all  players  are  using  Markovian  strategies. 
Yet,  this  equilibrium  is  not  intuitive  in  the  sense  that  it  is  resulting  from  an  artificial  coordination 
failure  among  A,B,  and  C. 

The  main  result  for  this  application  is  given  in  the  following  proposition  and  shows  how  the 
application  of  MTHPE  removes  the  non-intuitive  equilibrium  leading  to  {A,  B,  C,  D}. 

Proposition  1  Let  the  mapping  (p  be  defined  as 

4>  (X)  —  arg  min  7y.  (5) 

Y€{ZCN:  iz<ix;  lz>lx\z;  4>{Z)=Z}U{X} 

Then  there  exists  a  pure  strategy  SWUE  (MTHPE)  where  the  ultimate  ruling  coalition  is  reached 
after  at  most  one  stage  of  elimination  and  coincides  with  (p  {N)  with  probability  1,  and  the  payoff 
of  each  i  &  N  is  given  by 

Ui  (N)  =  Wi  {X)  -  el^N)  (i)  I{<f>{N):^N}-  (6) 

Vice  versa,  in  any  SWUE  in  pure  or  mixed  strategies  (and  in  any  MTHPE  in  pure  or  mixed 
strategies)  the  ultimate  ruling  coalition  is  (p  {N),  and  it  is  necessarily  reached  after  at  most  one 
stage  of  elimination,  and  players '  payoffs  are  given  by  (6) . 

Proof.  Acemoglu,  Egorov,  and  Sonin  (2008)  considered  a  version  of  this  game  with  sequential 
voting  and  established  that  the  unique  SPE  is  characterized  by  mapping  (5) ,  with  payoffs  (6)  as 
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given  in  the  proposition.  Given  the  notion  of  regularization  introduced  here,  this  impHes  that 
they  considered  a  regularization  of  the  game  here.  Then  Theorem  5  immediately  applies  and 
establishes  that  the  unique  MTHPE  is  given  by  (5)  and  (6),  completing  the  proof.    ■ 

This  proposition  provides  an  illustration  of  the  usefulness  of  the  concept  of  MTHPE.  Instead 
of  considering  a  more  complicated  game  with  sequential  voting  as  in  Acemoglu,  Egorov,  and 
Simon  (2008),  one  can  consider  a  simpler  game  and  obtain  exactly  the  same  results  (eliminating 
all  non-intuitive  equilibria). 

5      Conclusion 

In  many  political  economy  models,  which  involve  voting  or  multilateral  bargaining,  or  more 
generally,  in  what  we  defined  as  agenda-setting  games,  several  Nash  or  subgame  perfect  (Nash) 
equilibria  are  "non-intuitive" .  In  addition,  details  concerning  voting  and  proposal  making  pro- 
cedures often  have  a  major  effect  on  the  set  of  equilibria.  In  many  applied  models,  non-intuitive 
equilibria  are  eliminated  (sometimes  without  specifying  the  exact  notion  of  equilibrium),  and 
modelers  often  have  to  make  sure  that  the  "right  assumptions"  are  imposed  so  that  unimportant 
details  do  not  influence  conclusions. 

In  this  paper,  we  proposed  two  related  equilibrium  refinements  for  voting  and  agenda-setting 
games.  Sequentially  Weakly  Undominated  Equilibrium  (SWUE)  and  Markov  Trembling  Hand 
Perfect  Equilibrium  (MTHPE).  These  concepts  are  widely  apphcable,  easy  to  use  and  eliminate 
non-intuitive  equilibria  that  arise  naturally  in  dynamic  voting  games  and  games  in  which  random 
or  deterministic  sequence  of  agenda  setters  make  offers  to  several  players.  We  also  provided  a 
tight  characterization  of  MTHPE  in  agenda-setting  games  and  showed  that  the  set  of  MTHPE 
does  not  depend  on  features  such  as  whether  voting  is  simultaneous  or  sequential.  We  believe 
that  these  concepts  can  be  applied  in  a  wide  variety  of  political  economy  models  and  can  both 
eliminate  the  need  to  devise  ad  hoc  equilibrium  refinements  to  remove  non-intuitive  equilibria 
and  ensure  that  major  predictions  do  not  depend  auxiliary  assumptions  that  are  difficult  to  map 
to  reality  (such  as  whether  voting  is  simultaneous  or  sequential). 
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Appendix 

Example  5  Consider  the  following  extensive-form  game.  In  this  example,  player  3  gets  payoff 
0  in  all  terminal  nodes.  However,  his  choice  of  action  may  aflfect  the  payoffs  of  players  1  and 
2.  First,  players  1  and  2  simultaneously  choose  their  strategies.  (We  can  think  about  this  as 
a  simultaneous  vote.)  If  they  play  strategies  R  and  r,  respectively,  then  both  get  the  payoff  of 
2.  Otherwise,  player  3'  choice  determine  whether  they  both  get  1  or  0.  One  would  expect  that 
players  1  and  2  should  play  R  and  r,  respectively.  This  is  indeed  what  happens  in  any  MTHPE. 
(There  are  multiple  MHTPEs  as  player  3  is  indifferent  between  the  outcomes). 

Now  suppose  that  players  1  and  2  choose  their  actions  sequentially  (which  corresponds  to 
sequential  process  of  casting  votes).  If  they  play  (iJ, r),  both  get  the  payoff  of  2,  and  this 
is  arguably  the  most  intuitive  outcome.  However,  the  following  unintuitive  SPE  cannot  be 
eliminated  by  sequential  elimination  of  weakly  dominated  strategies.  Consider  the  strategy- 
profile  (I/, /,  01,62,63),  which  is  an  SPE.  None  of  player  3's  strategies  is  weakly  dominated. 
Then,  neither  L  nor  I  can  be  eliminated. 

Example  6  Consider  a  game  of  three  players  with  extensive  form  and  payoffs  as  shown  on 
Figure  4.  The  first  two  players  vote,  and  if  both  vote  for  the  'right',  all  three  players  receive 
first-best;  if  one  of  them  votes  for  the  'left'  then  the  third  player  chooses  between  'moderate' 
and  'bad'.  All  players  receive  the  same  in  all  terminal  nodes,  so  there  is  no  strategic  conflict 
between  them. 

Equilibrium  (i?,  r,  (ai,  02,03))  is  trembling-hand  perfect,  but  so  is  (Z/,Z,  (ai,  02,03))  where 
efficiency  is  not  achieved  because  of  'herding'  in  voting  (note  that  neither  L  not  I  are  dominated 
strategies:  for  instance,  L  is  best  response  to  second  player  playing  /  and  third  player  playing 
(01,62,63)).  Indeed,  take  some  t]  and  consider 

a"  =  ((1  -  7?3)  L  +  rj^R,  (l  -  rj^)  I  +  rj^r,  ((l  -  i?^)  oi  +  77^61,  (1  -  77)  02  +  7762,  (1  -  r?)  03  +  7763))  . 
Evidently,  player  3  (and  all  his  agents  in  agent-strategic  form)  are  better  oflF  choosing  oi  over 
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(5,5,5)  (0,0,0)        (5,5,5)  (0,0,0)        (5,5,5)  (0,0,0) 


(7,7,7) 


Figure  4:  A  Game  With  Herding  in  Trembling-Hand  Perfect  Equilibrium. 

6i,  a2  over  62,  and  03  over  63.  Now  consider  payoffs  of  player  1  choosing  L  or  R.  If  he  chooses 
L,  he  obtains  u/,  =  5  ((l  -  7?^)  (l  -  77^)  +rf  {I  -  r]))  =  5  -  bif  -  Sr;^  +  brf.  If  he  chooses  R, 
he  obtains  ur  =  b  ((l  —  77'^)  (1  —  77))  +  777^  =  5  —  577  +  277^  +  677^.  Hence,  For  small  77,  player 
1  should  put  all  weight  to  L,  and  a  similar  argument  would  show  that  player  2  should  put  all 
weight  to  I.  This  proves  that  (L,  /,  (ai,  02, 03))  is  also  a  trembling-hand  perfect  equilibrium. 

The  effect  that  Example  6  emphasizes  would  not  be  the  case  if  fully  mixed  profiles  cr"  were 
required  to  be  Markovian,  which  is  what  our  definition  of  MTHPE  imposes.  Indeed,  it  is  a 
natural  restriction  to  require  that  in  the  three  subgames  where  player  3  moves  and  payoffs  are 
identical,  his  mixed  action  profile  cr"  should  lead  to  identical  place.  In  that  case,  the  increase  of 
utility  of  player  1  due  to  the  possibility  of  player  2  playing  r  instead  of  I  would  be  not  be  offset 
by  worse  development  in  the  subgame  if  he  still  plays  /. 
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